Evaluation of the implementation of the objective structured clinical examination in health sciences education from a low‐income context in Tunisia: A cross‐sectional study

Abstract Background Objective structured clinical examination (OSCE) is well‐established and designed to evaluate students' clinical competence and practical skills in a standardized and objective manner. While OSCEs are widespread in higher‐income countries, their implementation in low‐resource settings presents unique challenges that warrant further investigation. Aim This study aims to evaluate the perception of the health sciences students and their educators regarding deploying OSCEs within the School of Health Sciences and Techniques of Sousse (SHSTS) in Tunisia and their efficacity in healthcare education compared to traditional practical examination methods. Methods This cross‐sectional study was conducted in June 2022, focusing on final‐year Health Sciences students at the SHSTS in Tunisia. The study participants were students and their educators involved in the OSCEs from June 6th to June 11th, 2022. Anonymous paper‐based 5‐point Likert scale satisfaction surveys were distributed to the students and their educators, with a separate set of questions for each. Spearman, Mann–Whitney U and Krusakll–Wallis tests were utilized to test the differences in satisfaction with the OSCEs among the students and educators. The Wilcoxon Rank test was utilized to examine the differences in students' assessment scores in the OSCEs and the traditional practical examination methods. Results The satisfaction scores were high among health sciences educators and above average for students, with means of 3.82 ± 1.29 and 3.15 ± 0.56, respectively. The bivariate and multivariate analyzes indicated a significant difference in the satisfaction between the students' specialities. Further, a significant difference in their assessment scores distribution in the practical examinations and OSCEs was also demonstrated, with better performance in the OSCEs. Conclusion Our study provides evidence of the relatively high level of satisfaction with the OSCEs and better performance compared to the traditional practical examinations. These findings advocate for the efficacy of OSCEs in low‐income countries and the need to sustain them.

examinations.These findings advocate for the efficacy of OSCEs in low-income countries and the need to sustain them.

K E Y W O R D S
health sciences, low-resource settings, medical education, objective structured clinical examination, reliability

| INTRODUCTION
Assessing student learning capabilities is crucial to improving healthcare education by measuring students' knowledge and skills and contributing to their learning through instructional strategies.In recent years, a growing recognition has emerged regarding the need to align assessment practices with the evolving demands of healthcare education, which increasingly emphasizes the development of practical skills and competencies alongside theoretical knowledge. 1,2Such alignment supports the integrity of academic accreditation and elevates the quality of education.This recognition suggests the need for a judicious selection of assessment strategies.
Consequently, educators ensure a range of evaluation approaches that accurately measure students' cognitive capabilities and, as importantly, psychomotor and other professional skills.
For several decades, the objective structured clinical examination (OSCE) has emerged as a gold standard for appraising medical students, particularly as they conclude their clinical rotations. 3The OSCE is generally characterized by its rigorous methodology, reliability, and validity. 4This examination approach helps bridge the gap between theoretical learning and practical application, offering a simulated environment to evaluate multifaceted clinical competencies. 5The OSCE thereby fosters a shift from rote memorization to acquiring hands-on skills indispensable for proficient healthcare delivery.[8][9] The COVID-19 pandemic has created various limitations in conducting practical assessments in medical education, leading health sciences educators worldwide to recalibrate their traditional educational methods.In response, Tunisia's Ministry of Higher Education has been at the forefront in catalyzing a paradigm shift towards more flexible and resilient educational strategies, including incorporating online simulations in OSCEs for the first time in Tunisia. 10 In Tunisia, the medical sciences education is organized under the License-Master-Doctorate (LMD) system.This system covers various medical education specialities, such as nursing.It also covers various health sciences educational programs, including Emergency Medical Care (EMC), Anesthesia Technologists (AT), Radiology Technologists (RT), Biology Technologists (BT), Surgical Technologists (ST), Paediatric Care (PC) and podologists, among other specialities. 11,12wever, the LMD has not yet been implemented in medicine, dentistry, and pharmacy. 11,12In Tunisia, the students allowed to continue in health sciences educational specialities programs are those who have successfully passed the national baccalaureate exam in experimental sciences or mathematics branches with respectable marks.These programs provide opportunities for capable students who could not secure admission or chose alternative paths beyond medicine, dentistry, or pharmacy programs yet still show a solid academic aptitude in their fields.In the Tunisian governmental medical education system, there are four faculties of medicine for medicine education, one faculty of dentistry for dentists, one faculty of pharmacy for pharmacists, four nursing institutes for nurses education and four Schools of Health Sciences and Technologies (SHST) for health sciences education.Each SHST is located near the biggest academic hospitals in Sousse, Monastir, Sfax and Tunis.They cover all the health sciences educational specialities (EMC, ST, RT, BT, and PC, among others).
Given the transformative trajectories that medical education will likely undertake in the post-pandemic era, forward-looking strategies are exigent.In the same context, OSCEs have recently been introduced into Tunisia's health sciences curricula.Hence, the perception of health sciences students and educators has never been measured.While OSCEs are universally recognized, little empirical inquiry exists to examine the feedback from health sciences students and educators from low-resource settings like Tunisia.Although previous studies in Tunisia have evaluated the use of OSCEs in specific domains, such as medical internships for medical students, 10,13 an assessment of its application across multiple health sciences educational programs has been lacking.This study contributes to the limited literature by evaluating the adoption of OSCEs in health sciences education at the SHSTS of the University of Sousse-a low-resource setting where constraints and educational traditions may pose unique challenges. 14

| Study design and setting
In the SHSTS, the OSCE is implemented as a comprehensive 1-day examination.It is designed to evaluate health sciences students' skills and clinical capabilities in a rigorous, structured, reliable, and valid manner.In the SHSTS, the OSCE comprised five to seven stations, according to the students' specialities.Each station was run in a separate room for over 7 min.The students could read an instruction sheet on the door of each room, briefing them about the station's background and skill being assessed.An educator was in the room to assess the student's performance according to a pre-defined evaluation grid without communicating any information.Once the student had finished performing the required task or reached 7 min without fully demonstrating the skill, the educator stopped the student and asked them to move quickly to the next station, as described in the summative OSCE approach in the article by Alinier. 5 The evaluation grid included critical and non-critical elements.Failure to perform one of the critical steps led to the student repeating the station another day after receiving feedback from the evaluator at the end of the OSCE examination day.
Anonymous paper-based 5-point Likert scale satisfaction surveys were distributed to the health sciences students and educators, with a separate set of questions for each (Appendixes 1 and 2).
Participation in the study was entirely voluntary.Students were provided with a consent form that clearly stated they could choose whether or not to participate without any consequences.Those who wished to participate signed the consent form.For those who declined, there were no repercussions.To protect the privacy and confidentiality of participants, all responses to the surveys were anonymised.This ensured that students could provide honest feedback without fear of potential repercussions.Both surveys (Annex 1 and 2) included 6 demographic questions, 17 questions about the particularity of the OSCE, 10 about the structure, 9 about the organization, and 5 about its efficiency (validity and reliability) in assessing the students' skills.
In addition, all health sciences students participating in the OSCE underwent a 2-h practical exam 2 weeks after the OSCE assessment dates.This exam involved practising a care skill in a real-world scenario, where students interacted with actual patients and healthcare personnel in relevant departments, complementing the structured assessment of the OSCE.An educator was present during the practical exam to observe the students.While they did not interfere with the student's interactions, they assessed performance using a predefined checklist for each care task.The care tasks assigned to each student were randomly determined based on the clinical presentation of the patient and the student's health science educational specialities (e.g., Arterial blood gas test, Supra glottic and endotracheal airway control for EMC, and assisting in surgery by passing tools and retracting tissues for ST).

| Participants and sampling
The study included the third-year SHSTS all specialities students (the graduation year) (N = 133) and their educators (N = 33).Slovin's formula was utilized to determine the minimum sample size required: 98 for the students and 31 for the educators.

| Data analysis
IBM-SPSS version 26 was utilized for data analyzes.First, both surveys were validated using the Aiken V content validity coefficient (CVC) to determine whether the surveys measured what they intended to measure. 15Five experts in medical research and OSCE training were invited.A letter (Annex 3) explaining the study's objective was sent to them.These experts were asked to rate each survey item on a scale from 1 (lowest) to 5 (highest) for pertinence, clarity, and how well the item served as a good indicator of the intended measure.Then, based on their scoring, the CVC was calculated.Second, Cronbach alpha for reliability analysis was also determined.It aimed to determine whether or not we might get the same results if the surveys were repeated on another population with the same characteristics and under the same conditions.Third, descriptive statistics were conducted.The average of the satisfaction scores for each item was determined.Fourth, bivariate and multivariate analyzes were performed.The quantitative variables' Gaussian distribution was verified using Shapiro and Kolmogorov test.Then, accordingly, the Spearman tests were conducted to test the scores' correlation between the variables.The Man-Whitney U test was performed to test the distribution of the satisfaction score within each health sciences student group.Furthermore, the Wilcoxon signed ranks test was utilized to evaluate whether there were statistically significant differences between the scores of traditional practical examinations and the average score of the health sciences students in the various sections.
The Kruskal-Wallis test was performed to test the following hypotheses: Then, based on the Kruskal-Wallis test results, the post Table 1 presents demographic information of the participating population.
The CVCs of the health sciences students and educators' surveys were determined.They were respectively equal to 0.71 ± 0.44 (Table 2) and 0.82 ± 0.02 (Table 3), indicating a solid validity of both tools.The reliability of both tools was also assessed in Table 4, giving Cronbach alpha coefficients for both tools equal to 0.96 and 0.83, indicating the solid reliability of both tools and that they would give the same results if repeated under the same circumstances.
The average overall satisfaction scores of health sciences students and educators were determined and represented in the Shewhart charts in Figure 1.In Shewhart charts, the control limits are calculated based on the data and represent the natural boundaries within which the process is considered stable and desirable. 16The upper control limit (UCL) and lower control limit (LCL) are set at three standard deviations above and below the mean, respectively.In Figure 1A, the educators' satisfaction scores varied mostly within the control limits, indicating a stable process.
Furthermore, the mean satisfaction score for educators was above 3, which is considered a high satisfaction level based on the 5-point Likert scale used in the survey.In Figure 1B, the students' satisfaction scores varied mainly within the control limits, suggesting an overall stable process.However, the mean satisfaction score for students fluctuated between high (≥3) and low (<3) levels on the 5-point Likert scale, indicating more variability in their satisfaction compared to the educators.
The quantitative variable distributions (students' ages, health sciences educators' ages, years of experience, and satisfaction scores) were verified using the Shapiro test.All variables' p-values were <0.05, indicating that these quantitative variables were not normally distributed.Table 3 indicates the bivariate and multivariate analysis results.
Further, in Table 5, for the health sciences educators, there was a fair, positive correlation between the particularity and structure of the OSCE and a strong positive correlation between the age and experience of the health sciences educators and the overall satisfaction scores.The Kruskall-Wallis and post hoc tests indicated a significant difference in the satisfaction distribution between the health sciences educators' groups according to their backgrounds.It is worth mentioning here that the health sciences educators' backgrounds were classified according to their specialities before they underwent the pathway of health sciences education.The post hoc test indicated the highest satisfaction among educators who previously studied EMC and ST.
Additionally, in Table 6, for the students, there is a strong positive correlation between the students' satisfaction variables.Kruskall-Wallis and post hoc tests demonstrated that EMC students were more satisfied than the ST, followed by PC and podology students.
Finally, the results in Table 7 indicate significant differences in scores between the practical examinations and OSCE in the EMC, ST, and PC sections, with OSCE scores tending to be higher.No significant difference was found in the podologists section.The descriptive statistics provide additional insights into the average scores and their distribution within each section.

| DISCUSSION
In a controlled and reproducible environment, OSCEs offer a standardized, objective, and comprehensive approach to assessing clinical skills, including communication, history-taking, physical examination, and clinical reasoning.This structured assessment method has gained significant traction in medical and allied health education globally, as it provides a more reliable and valid measure of clinical competence compared to traditional assessment methods, such as written examinations or unstructured clinical evaluations.
The present study identified various insights regarding the students' and educators' perception of OSCEs, particularly in a Tunisian low-resource environment like the SHSTS.Our findings demonstrated the robust validity and reliability of OSCEs, echoing previous research that has established OSCEs as a reliable and positively appreciated assessment strategy in healthcare education. 3,5,17,18This validation is crucial, especially in a Tunisian context where new methods are introduced, signaling a potential shift in healthcare education assessment within low-resource settings.The demographic findings revealed that satisfaction scores were high among educators but showed more variance among students.This discrepancy in satisfaction rates could reflect a range of factors, including the novelty of OSCEs in Tunisia, varying expectations, and differing levels of familiarity and stress associated with this assessment format.One approach to consider is organizing formative OSCEs so educators and students can become more acquainted with this assessment approach.An additional way of dissipating the students' stress associated with potentially underperforming in a given station is to ensure that an OSCE is constituted of a higher number of stations, sometimes including "theoretical stations," hence ensuring that each skill is assessed several times in different ways and contexts. 8,19This also contributes to increasing the validity and reliability of the overall assessment approach.Such a divergence in satisfaction rates between these two groups warrants further investigation into the underlying causes of the pedagogical backgrounds and training.They can profoundly influence satisfaction levels, a crucial metric in educational quality.This aligns with the traditional didactic method, characterized by lecturer-led teaching.While this approach has long been considered essential, its efficacy in meeting contemporary educational needs in all environments is increasingly questioned. 20Studies show that didactic learning may need to improve in fostering critical thinking and practical skills while efficiently disseminating factual knowledge, potentially leading to lower satisfaction levels among students who request a more interactive and engaging learning environment. 17,21In contrast, problem-based learning, such as the OSCE, represents a suitable change, emphasizing student-centered learning and practical problem-solving skills. 22Problem-based learning has been shown to enhance student satisfaction by actively engaging learners in the educational process, promoting more profound understanding, and fostering critical thinking.However, the success of problem-based learning is contingent upon the health sciences educators' ability and willingness to facilitate rather than direct learning, which can be a significant cultural and pedagogical shift for faculty accustomed to traditional methods. 23Integrating simulation-based training provides a safe, controlled environment for practising clinical skills, 19 which, in our study, has been positively correlated with both student and educator satisfaction due to its practicality and relevance to clinical practice.

Age
Moreover, the positive perceptions of Tunisian health sciences educators and students suggest a readiness and willingness to embrace this assessment approach, which could drive educational The need for quality improvement in healthcare education is pressing, and standardized, internationally recognized assessment methods like the OSCE would be crucial in driving educational reform.This change improves assessment practices and enhances educational objectives, ensuring that teaching methods and student performance evaluations are coherent, comprehensive, and conducive to producing competent healthcare professionals.
Introducing OSCEs in Tunisia represents a ground-breaking move towards aligning the country's medical education with global standards, particularly in low-resource contexts.The need for quality improvement in healthcare education is pressing, and standardized, internationally recognized assessment methods like the OSCE could be crucial in driving educational reform.This change improves assessment and enhances educational objectives, ensuring that teaching methods and student performance evaluations are coherent, comprehensive, and conducive to producing competent healthcare professionals. 24Globally, OSCEs are increasingly recognized for their utility in diverse educational contexts.In higher-income countries like the United Kingdom and the United States, OSCEs are integral to medical and nursing education. 25,26Their adoption in lower-middle-income countries, including India and Nigeria, indicates a growing acknowledgment of their effectiveness. 27,28rthermore, the findings presented in this study, which indicate that students achieved better scores in the OSCE compared to traditional practical assessments, highlight the potential benefits of this assessment approach in accurately evaluating clinical competencies.The controlled and standardized nature of OSCEs mitigates biases inherent in real-world practical examinations, where factors such as patient conditions and reactions may distract students from focusing on the assessed skills.By providing a simulated yet realistic environment, OSCEs enable a more objective and focused evaluation of students' abilities, potentially leading to more accurate and reliable assessments of their readiness for clinical practice.
However, the introduction of OSCEs in new contexts like Tunisia is challenging.Institutional traditions, health sciences educators' capabilities, and students' familiarity are barriers that must be considered.Our study is a foundational step in this direction, providing a model that can be adapted and refined for broader implementation.Furthermore, the results presented in Table 7 indicate that most students achieved better scores in the OSCE compared to the traditional practical assessment.This latter method relies on direct interaction with patients to perform clinical skills pertinent to health sciences practice, with students receiving F I G U R E 1 Shewhart chart for students' (B) and educators' (A) average satisfaction scores.
assistance from full-time clinical personnel in the department.In such assessments, the health sciences educators must remain impartial.
However, while offering real-time feedback, this approach can be biased due to various factors, such as patient conditions and reactions.These may distract the students from focusing on the assessed skills, potentially impairing their performance.Moreover, in low-income countries like Tunisia, where continuous education for health sciences personnel is not as well-established as in countries such as Finland, the United Kingdom, and other countries, [29][30][31] the regular staff's commitment to best practices might be compromised, consequently affecting students' performance and resulting in lower scores on the evaluation grid.In contrast, the OSCE method simulates a real-life scenario while challenging the student to complete the skill within a designated timeframe, free from the biases inherent in a real practical examination.Therefore, using the OSCE as an evaluation method is recommended over the traditional practical examination format.
Further, high-quality healthcare education is a global imperative, yet access to it in low-income countries needs improvement in the delivery and assessment. 32 in the region across similar low-income settings.Moreover, the COVID-19 pandemic has highlighted the importance of adaptable, safe educational practices.The pandemic has highlighted the challenges faced by resource-limited healthcare systems, emphasizing the need for robust healthcare education in such contexts. 33,34e pandemic has also emphasized the importance of adaptable and resilient educational strategies, including incorporating online simulations and virtual OSCEs. 35As SHSTS at the University of Sousse, Tunisia, is a pioneer in implementing OSCEs in health sciences education within the country, SHSTS's proactive response in catalyzing a culture of resilient educational strategies during the COVID-19 pandemic demonstrates a solid commitment to embracing innovative approaches and ensuring the continuity of high-quality healthcare education despite the unprecedented challenges posed by the public health crisis.Our study's relatively high satisfaction levels showed that the OSCEs offer a controlled environment for clinical competence assessment. 36,37 conclusion, our study corroborates the high levels of validity

| LIMITATIONS
The study was unicentric, restricting the generalizability of the  | 15 of 15 By examining the perceptions and experiences of students and educators across various health sciences specialities, the research offers insights into implementing and accepting OSCEs within Tunisian health sciences education in other institutions.The multidisciplinary approach provides a broader perspective on the feasibility and potential barriers to integrating OSCEs across different health sciences programs in resource-limited environments.This study aims to evaluate the health sciences students' and healthcare educators' perception of deploying OSCEs within the SHST of Sousse (SHSTS) at the University of Sousse in Tunisia and its perceived efficacity in medical education compared to the traditional practical examination methods.This cross-sectional study was conducted in June 2022 for the SHSTS final-year students in Sousse who had undergone the OSCE and the classic practical examinations in May and June 2022.The health sciences educational specialities in the SHSTS are EMC, ST, PC and Podology.The article's structure adopts the Consolidated Standards of Reporting Trials (CONSORT) checklist.Ethical approval for this study was obtained from the Faculty of Medicine "Ibn Eljazzar" Doctoral School of the University of Sousse review board on 27/03/2021.
hoc test was conducted to determine which samples had different satisfaction distributions.Shewhart chart was performed to analyze the variation of the satisfaction scores across the group.BEN AMOR ET AL. | 3 of 15 3 | RESULTS A total of 131 students and 33 educators participated in the surveys.
and reliability of OSCEs and survey instruments, aligning with existing research.The relatively high satisfaction levels reported among health sciences educators and students in the SHSTS, coupled with the demonstrated efficacy of OSCEs in accurately assessing clinical competencies, highlight the potential for this assessment approach to drive transformative change in the health sciences education at the University of Sousse in Tunisia, contributes to the development of a more competent and wellprepared healthcare workforce, capable of delivering high-quality patient care in resource-limited settings.It addresses a critical literature gap by setting its research in a low-income country, providing empirical evidence of the need to adapt medical education techniques for resource-limited settings.
Students' Aiken V validity coefficient results.Health sciences educators'Aiken V validity coefficient results.
OSCEs in Tunisian health sciences education.The successful implementation of OSCEs in Tunisia represents a groundbreaking move towards aligning the country's medical education with international best practices, particularly in resource-limited settings.T A B L E 3 Bivariate and multivariate analysis results for the Health sciences educators.Bivariate and multivariate analysis results for the senior health technology students.
Integrating reliable and valid assessment methodologies like the OSCE into settings such as Tunisia is significant.It represents a move towards global best practices in healthcare education, potentially driving quality improvement and boosting the pursuit of educational excellence and healthcare quality T A B L E 5 a Pearson correlation.b Spearman correlation (Rho).c Mann-Whitney U test.d Kruskal-Wallis Test.e Post hoc.T A B L E 6 a Spearman correlation (Rho).b Mann-Whitney U test.c Kruskal-Wallis test.d Post hoc.
The descriptive and Wilcoxon Rank test for the practical and OSCE tests' results.